Specific person detection system and specific person detection method

ABSTRACT

This specific person detection system: identifies, from among the persons recorded in a specific person recording unit, a person who most closely matches a feature value extracted from image data; calculates the degree to which feature values of a plurality of persons extracted from other image data match the identified person; and outputs, as an identification result, information about a person who has a feature value closely matching the identified person, and who is associated with angle information that most closely matches angle information associated with the identified person.

TECHNICAL FIELD

The present invention relates to a specific person detection system, andis applicable to, e.g., to a specific person detection system to detecta specific person using coincidence obtained from an image featurevalue.

BACKGROUND ART

Systems and techniques for automatically detecting occurrence ofspecific event in a video image in a real-time manner, using an imagerecognition technique, are developed. A specific person detection systemis known as typical one of such systems. The specific person detectionsystem is a system having a function of automatically detecting a personpreviously image-registered as a detection object (hereinafter, specificperson), when appears in a monitored video image, and notifying a systemoperator of the detection of the person. The detection is performed bycollation of image feature value between a specific person's image andimages of a person appeared in the monitored video image (appearedperson), to calculate coincidence between both persons' images, and whenthe coincidence is a value within a certain range, determining that theyare the same person, i.e., the specific person.

CITATION LIST Patent Literature

PTL 1: Japanese Patent Application Laid-Open No. 2014-106816

SUMMARY OF INVENTION Technical Problem

In the specific person detection system, the appeared person is notaware of the presence of the camera in many cases in accordance with thecharacteristic of the monitoring camera. As the appeared person does notalways face the camera, the image feature value is not stabled due todirection fluctuation of the person in some cases. This causesdegradation of detection accuracy in the specific person detectionsystem. More particularly, this appears as overlooking and erroneousdetection.

To address the direction fluctuation, preparation and registration ofimages from various angles in advance regarding a previously-registeredperson's image to improve the accuracy is proposed (e.g., JapanesePatent Application Laid-Open No. 2014-106816). However, in some cases,it is difficult to previously prepare images from various angles.

The object of the present invention is to provide a specific persondetection system in which the degradation of detection accuracy isreduced.

Solution to Problem

In the present disclosure, the outline of a typical aspect will bedescribed as follows.

That is, in a specific person detection system, a person recorded in aspecific person unit most similar to a feature value extracted fromimage data is obtained. Similarity between a feature value extractedfrom other image data and the person is calculated. Among persons withhigh similarity, a person having angle information most similar to theperson is outputted as a collation result.

Advantageous Effects of Invention

According to aspects the present invention, it is possible to obtainhigher detection accuracy in the specific person detection system.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a device configuration of a specific persondetection system according to an example 1.

FIG. 2 is a diagram showing a functional configuration of a detectiondevice in FIG. 1.

FIG. 3 is a diagram showing an aspect of record data in a specificperson recording unit in FIG. 2.

FIG. 4A is a diagram showing the flow of processing in the detectiondevice in FIG. 1.

FIG. 4B is a diagram showing the flow of processing in the detectiondevice in FIG. 1.

FIG. 4C is a diagram showing the flow of processing in the detectiondevice in FIG. 1.

FIG. 5 is a diagram showing an aspect of stored data in a determinationresult storing unit in FIG. 2.

FIG. 6 is a diagram explaining timing.

FIG. 7A is a diagram explaining the content of processing at time T.

FIG. 7B is a diagram explaining the content of processing at time T.

FIG. 8A is a diagram explaining the content of processing at steps 411to 420.

FIG. 8B is a diagram explaining the content of processing at steps 411to 420.

FIG. 9A is a diagram explaining the content of processing at steps 422to 432.

FIG. 9B is a diagram explaining the content of processing at steps 422to 432.

FIG. 10 is a diagram showing the device configuration of the specificperson detection system according to an example 2.

FIG. 11A is a diagram showing the flow of processing in the detectiondevice in FIG. 10.

FIG. 11B is a diagram showing the flow of processing in the detectiondevice in FIG. 10.

FIG. 11C is a diagram showing the flow of processing in the detectiondevice in FIG. 10.

FIG. 12A is a diagram explaining the content of processing at time T.

FIG. 12B is a diagram explaining the content of processing at time T.

FIG. 13A is a diagram explaining the content of processing at steps 1110to 1117.

FIG. 13B is a diagram explaining the content of processing at steps 1110to 1117.

FIG. 14A is a diagram explaining the content of processing at steps 1118to 1126.

FIG. 14B is a diagram explaining the content of processing at steps 1118to 1126.

FIG. 15 is a diagram showing the device configuration of the specificperson detection system according to an example 3.

FIG. 16 is a diagram showing positional relationship in installation ofimage pickup devices in a building in overhead view.

FIG. 17 is a diagram showing an aspect of distance data betweenimage-pickup areas in a control instruction unit 209.

FIG. 18A is a diagram showing the flow of processing in the detectiondevice in FIG. 15.

FIG. 18B is a diagram showing the flow of processing in the detectiondevice in FIG. 15.

FIG. 19A is a diagram explaining the content of processing at time T.

FIG. 19B is a diagram explaining the content of processing at time T.

FIG. 20A is a diagram explaining the content of processing at time T+18.

FIG. 20B is a diagram explaining the content of processing at time T+18.

FIG. 20C is a diagram explaining the content of processing at time T+18.

FIG. 21 is a diagram showing a functional configuration of a retrievaldevice in FIG. 15.

DESCRIPTION OF EMBODIMENTS

In a specific person detection system according to embodiments, a personrecorded in a specific person unit most similar to a feature valueextracted from image data is obtained. Similarity between a featurevalue extracted from other image data and the person is calculated.Among persons with high similarity, a person having angle informationmost similar to the person is outputted as a collation result.

According to the specific person detection system according to theembodiments, it is possible to obtain higher detection accuracy.

Hereinbelow, the embodiments will be described using the drawings. Notethat in the following description, the same constituent elements havethe same reference numeral. Repetitive explanations may be omitted.

Example 1

A device configuration of a specific person detection system accordingto an example 1 will be described using FIG. 1. In a specific persondetection system 10, an image pickup device 101, a recording device 102,a detection device 103, and a terminal device 104 are connected to anetwork 100 in a mutually communicable status.

The network 100 is a communication line such as a specialized line, anIntra-net, the Internet, or a wireless LAN, for mutually connecting therespective devices for data communication.

The image pickup device 101 is a device such as a monitoring camerahaving a zoom lens for perspective control and focus control, an imagepickup device such as a CCD or a CMOS, an A/D converter for digitalconversion of an analog signal, a temporary memory such as a RAM, a datatransmission bus, a timing circuit, an external input/output interface,a power source circuit, a camera platform for panning, tilting and thelike, an illumination device such as a visible light device or anear-infrared LED, and the like.

The image pickup device 101 converts light passed through the lens intoan electric signal with the image pickup device. The image pickup deviceconverts the converted electric signal into a digital value with the A/Dconverter. Then the image pickup device stores the converted digitalvalue into the temporary memory as image data. The stored image data isoutputted from the external input/output interface to the network 100 inaccordance with an external video image request inputted into theexternal input/output interface, an instruction from the timing circuit,or the like. Further, the direction of image pickup, the perspective andfocus of the lens, the quantity of illumination and the like are changedin accordance with an external control command similarly inputted intothe external input/output interface.

The recording device 102 is a device such as a network digital recorderhaving a temporary memory such as a RAM, a recording medium such as anHDD, a data transmission bus, a timing circuit, an external input/outputinterface, a power source circuit, and the like.

The recording device 102 records image data from the image pickup device101, inputted from the network 100 to the external input/outputinterface, with its image time, into the recording medium such as anHDD. The recorded image data is outputted from the external input/outputinterface to the network 100 in accordance with an external video imagerequest inputted in the external input/output interface, an instructionfrom the timing circuit or the like. The video image request includesimage time. The recording device 102 outputs image data at image timeincluded in the video image request.

In the present example, the recording device 102 always records thevideo image from the image pickup device 101.

The detection device 103 is a device such as a computer having anarithmetic circuit such as a CPU, a temporary memory such as a RAM, arecording medium such as an HDD, a data transmission bus, an externalinput/output interface, a power source circuit, and the like.

The detection device 103 stores image data from the image pickup device101, inputted from the network 100 in the external input/outputinterface, into the temporary memory. The detection device performsvarious arithmetic operations related to specific person detection,using the arithmetic circuit, on the stored image data. The recordingmedium holds a set of software application programs, an OS (OperationSystem), a set of specific persons' images (person feature values), andthe like. The result of specific person detection is outputted from theexternal input/output interface to the network 100.

The terminal device 104 is a device such as a computer having anarithmetic circuit such as a CPU, a temporary memory such as a RAM, arecording medium such as an HDD, a data transmission bus, an externalinput/output interface, a power source circuit, a screen such as aliquid crystal display, and user input devices such as a keyboard and amouse. Further, the terminal device 104 may be additionally providedwith e.g. a buzzer or voice announcement device, an LED and the like.

The terminal device 104 stores image data from the image pickup device101, inputted from the network 100 in the external input/outputinterface, and a specific person detection result from the detectiondevice 103, into the temporary memory. The terminal device converts thestored image data and the specific person detection result, using thearithmetic circuit, in a format appropriate for display, and displaysthem on the screen. The recording medium holds a set of softwareapplication programs, an OS, and the like. Further, user operation withrespect to the terminal device 104 is performed with respect to the userinput device.

As shown in FIG. 2, the detection device 103 is configured withrespective processors, a specific person recording unit 200, an imagereception unit 201, a person detection unit 202, a person's anglecalculation unit 203, a person's feature value calculation unit 204, aspecific person determination unit 205, a determination result storingunit 206, a specific person comprehensive determination unit 207, adetection notification unit 208, and a control instruction unit 209.

The specific person recording unit 200 is a processing unit which, uponimage registration of a specific person, records the person's image, theperson's feature value, the person's ID (name, identificationinformation uniquely given to the person) and the like in the recordingmedium. For example, it is a database. It is configured such that thedata is previously given, in any way. The aspect of the record data inthe specific person recording unit will be described later.

The image reception unit 201 is a processing unit which performs imageinput/output from the outside of the device. In the present example, itreceives input image data from the image pickup device 101 and therecording device 102.

The person detection unit 202 performs person detection using an imagerecognition technique with respect to the input image data in the imagereception unit 201, determines the existence of person. When a personexists in the image, the person detection unit 202 outputs thecoordinates of the region.

The person's angle calculation unit 203 is a processing unit whichperforms angle calculation using the image recognition technique withrespect to the person's region in the image detected with the persondetection unit 202.

The person's feature value calculation unit 204 is a processing unitwhich performs feature value calculation using the image recognitiontechnique with respect to the person's region in the image detected withthe person detection unit 202.

The specific person determination unit 205 is a processing unit whichperforms primary determination as to whether or not the person detectedwith the person detection unit 202 corresponds to a specific person,using the feature value obtained with the person's feature valuecalculation unit 204.

The determination result storing unit 206 is a processing unit whichstores the result of determination with the specific persondetermination unit 205 and the like into the temporary memory and savesthem. The aspect of the stored data in the determination result storingunit will be described later.

The specific person comprehensive determination unit 207 is a processingunit which performs secondary determination with respect to thedetermination result, determined with the specific person determinationunit 205 and stored in the determination result storing unit 206, fromplural determination results.

The detection notification unit 208 is a processing unit which, upondetection of a specific person with the specific person comprehensivedetermination unit 207, performs input/output of the resultnotification. It transmits specific person detection result data to theterminal device 104.

The control instruction unit 209 is a processing unit which issues aninstruction to the respective processing units.

As shown in FIG. 3, in record data 320 in the specific person recordingunit 201, e.g. information on one person is recorded as one record.Information on five specific persons, i.e. five records are recorded asrecords 300 to 304.

The records 300 to 304 are respectively configured with a record numbercell 310, a person ID cell 311, a feature value cell 312, an angle cell313, and a person image cell 314.

The record number cell 310 is a region to store a record number. Therecord number is a number used for record management. For example, it isa continuous integer value uniquely assigned by record.

The person ID cell 311 is a region to store a person ID of a specificperson. In the present example, a character string as a person name isstored. As long as the ID is identification information uniquely givento the person, an integer value or the like may be used.

The feature value cell 312 is a region to store a person's feature valueof a specific person. In the present example, for the sake of simplicityof explanation, a small number value as a one dimensional value isstored. However, a multidimensional value may be used.

The angle cell 313 is a region to store an image pickup angle of aspecific person.

The person image cell 314 is a region to store a specific person'simage. In the present example, a person image itself is stored. However,an address of a region separately storing the image may be stored in theform of hexadecimal number.

In the present example, a set of a feature value, an angle, and an imageis stored per one person. It may be configured such that plural sets offeature amounts, angles, and images are stored per one person.

The flow of processing in the detection device 103 will be describedusing FIGS. 4A to 4C.

Step 400: the control instruction unit 209 calculates time (T) of animage to be obtained next. When specific person detection is performedby several frames, time several frames ahead of current time iscalculated as image time (T). Then, the image reception unit 201requests the image pickup device 101 to output an image (T).

Step 401: the image reception unit 201 performs image reception waiting.When image incoming from the image pickup device 101 is detected, theprocess proceeds to step 402.

Step 402: the image reception unit 201 receives an image from the imagepickup device 101. The image at the reception time (T) is the image (T).

Step 403: the person detection unit 202 detects person from the receivedimage data. The person detection is performed by e.g. a method of movingbody detection with a difference from a background image and furtherperforming person determination with area, shape, moving speed or thelike of its moving body region, a method of searching existence/absenceof region having an image feature pattern of a person, previouslylearned and prepared, using intra-image searching, or the like. In thepresent example, the person detection may be performed by any method.Further, the person detection may be performed on the entire person ormay be performed on a partial region such as a face which is a typicalportion in person specification. As a result of the above detection,when a person is detected, the process proceeds to step 404. When aperson is not detected, the process returns to step 400.

Step 404: the person detection unit 202 calculates coordinates person'sregion coordinates of the person detected at step 403.

Step 405: the person's angle calculation unit 203 calculates an imagepickup angle with respect to the person's region image calculated atstep 404. The calculation of person's image pickup angle is performed bye.g. a method of detecting respective organs such as eyes, a nose, amouth and the like from the region image, and obtaining the angle fromthe arrangement relationship among them, a method of previouslypreparing an image from an already known angle and obtaining the anglefrom similarity with the image, or the like. In the invention accordingto the present example, any method may be used.

Step 406: the person's feature value extraction unit 204 calculates aperson's feature value with respect to the person's region imagecalculated at step 404. As the calculated person's feature value, e.g.the shape or direction of the person's outline, colors of hair and skin,gait, or the shape or direction of the face outline, sizes and shapes ofprimary constituent elements such as the eyes, the nose and the mouth,arrangement relationship among them, and the like, are given. In thepresent example, any type(s) and/or number of feature value(s) may beused.

Step 407: the specific person determination unit 205 performs collationon all the person's feature values. More particularly, the person'sfeature value calculated at step 406 is collated (coincidencecalculation) sequentially with respect to all the persons' featurevalues of the respective specific persons previously recorded in thespecific person recording unit 200. A feature value with the highestcoincidence is found. A person having this feature value is determinedas a most similar person. Note that the coincidence is a numerical valueindicating proximity between images (image feature values). Regardingthe calculation method, e.g. papers such as “Visualization Models forLarge Image Sets” (HIROIKE, Atsushi et.al, Journal of Japan PhotoSociety 2003, vol. 66 no. 1, P 93-P 101) may be referred to. In thecoincidence in the present example, as the value is smaller, thecoincidence is higher. In the present example, as methods for thedetermination of the collation order and the calculation of thecoincidence, any method may be used.

Step 408: the specific person determination unit 205 performs specificperson detection determination (primary detection). As the detectiondetermination, regarding the most similar person obtained at step 407,in the relationship between the coincidence and a certain constant value(first threshold value), when the coincidence is equal to or lower thanthe threshold value, it is determined that the person has been detectedas the specific person. When detection is determined, the processproceeds to step 409, otherwise, the process returns to step 400.

Step 409: the determination result storing unit 206 stores the resultobtained with respect to the image (T) and saves it. Note that the dataas the object of storage includes the coincidence and the person ID ofthe most similar person obtained at step 408, the angle obtained at step405, and the coordinates of the person's region calculated at step 404.The aspect of the stored data will be described later.

Step 410: the control instruction unit 209 resets a counter (L) to 1.

Step 411: the control instruction unit 209 compares the counter (L) witha previously-set limit value (M). When the value L is less than thevalue M, the process proceeds to step 412. When the value L is equal toor greater than the value M, the process proceeds to step 421. Note thatthe value M is e.g. 4.

Step 412: the image reception unit 201 requests the recording device 102to output an image (T-L) L-frame before of the image (T).

Step 413: the image reception unit 201 receives the image (T−L) from therecording device 102.

Step 414: the person detection unit 202 performs detection of a personthe same as the person detected at step 403 with respect to the receivedimage data. At this time, when the counter (L) is 1, detection isperformed only on a neighboring region of the region image of the personcalculated at step 404, otherwise, calculated at previous step 414, asthe object of detection. The method for the person detection is as inthe case of step 403. The identification between the person detected atstep 403 and the person detected at the present step is made by, e.g. amethod of obtaining an overlap area between the person's region obtainedat step 403 and the person's region obtained at the present step, andwhen the value is equal to or higher than a constant value, they are thesame, or the like. The above constant value is previously given inconsideration of an apparent moving amount of the person in the image.As a result of the above operation, when the same person is detected,the process proceeds to step 415. When the same person is not detected,the process proceeds to step 421.

Step 415: the person detection unit 202 calculates the person's regioncoordinates of the person detected at step 414.

Step 416: the person's angle calculation unit 203 calculates an imagepickup angle with respect to the region image of the person calculatedat step 415. The method for angle calculation is the same as in the caseof step 405.

Step 417: the person's feature value extraction unit 204 calculates theperson's feature value with respect to the region image of the personcalculated at step 415. The method for calculation of person's featurevalue is the same as in the case of step 406.

Step 418: the specific person determination unit 205 performs collationof the person's feature value. More particularly, the feature value ofthe most similar person obtained at step 407 is read from the persons'feature values of the respective specific persons, previously recordedin the specific person recording unit 200. The read feature value iscollated with the person's feature value calculated at step 417, andcoincidence is calculated. The method for coincidence calculation is thesame as in the case of step 407.

Step 419: the determination result storing unit 206 stores the resultobtained with respect to the image (T-L) and saves it. Note that thedata as the object of storage includes the coincidence obtained at step418, the angle obtained at step 416, and the coordinates of the person'sregion calculated at step 415.

Step 420: the control instruction unit 209 increments the counter L by1, and returns the control to step 411. In the present example, theincrement amount is 1, i.e., continuous frames are used. When an imagepickup frame rate of the image pickup device 101 is high with respect tothe moving speed of a person in the image, the increment amount may be 2or more, to use discrete frames. The frames may be similarly used hereinbelow.

Step 421: the control instruction unit 209 resets the counter (L) to 1.

Step 422: the control instruction unit 209 compares the counter (L) witha previously-set limit value (N). When the value L is less than thevalue N, the process proceeds to step 423. When the value L is equal toor greater than N, the process proceeds to step 432. Note that N is e.g.4.

Step 423: the image reception unit 201 requests the recording device 102to output an image (T+L) L-frame after the image (T).

Step 424: the image reception unit 201 receives the image (T+L) from therecording device 102.

Step 425: the person detection unit 202 performs person detection withrespect to the received image data as to the same person as the persondetected at step 403. At this time, when the counter (L) is 1, detectionis performed only on a neighboring region of the region image of theperson calculated at step 404, otherwise, calculated at previous step425, as the object of detection. The method for person detection is thesame as in the case of step 403. The identification between the persondetected at step 403 and the person detected at the present step is thesame as in the case of step 414. As a result of the above operation,when the same person is detected, the process proceeds to step 426. Whenthe same person is not detected, the process proceeds to step 432.

Step 426: the person detection unit 202 calculates the person's regioncoordinates of the person detected at step 425.

Step 427: the person's angle calculation unit 203 calculates an imagepickup angle with respect to the region image of the person calculatedat step 426. The method for angle calculation is the same as in the caseof step 405.

Step 428: the person's feature value extraction unit 204 calculates aperson's feature value with respect to the region image of the personcalculated at step 426. The method for person's feature valuecalculation is the same as in the case of step 406.

Step 429: the specific person determination unit 205 performs collationon the person's feature value. More particularly, the feature value ofthe most similar person obtained at step 407 is read from the persons'feature values of the respective specific persons previously recorded inthe specific person recording unit 200. The read feature value iscollated with the person's feature value calculated at step 428, and thecoincidence is calculated. The method for coincidence calculation is thesame as in the case of step 407.

Step 430: the determination result storing unit 206 stores the resultobtained with respect to the image (T+L) and saves it. Note that thedata as the object of storage includes the coincidence obtained at step429, the angle obtained at step 425, and the coordinates of the person'sregion calculated at step 426.

Step 431: the control instruction unit 209 increments the counter (L) by1, and returns the control to the step 422.

Step 432: the specific person comprehensive determination unit 207performs comprehensive determination using the results stored in thedetermination result storing unit 206 at steps 409, 419, and 430. Thecomprehensive determination is performed by reading the image pickupangle of the most similar person obtained at step 407 from the imagepickup angles of the respective specific persons previously recorded inthe specific person recording unit 200, searching for a result of angleclosest to the angle, among the results stored in the determinationresult storing unit 206, and in the relation between the coincidencebetween the results and a certain constant value (second thresholdvalue), when the coincidence is equal to or lower than the thresholdvalue, comprehensively determining that the person has been detected asthe specific person. As the second threshold value, a value smaller thanthe first threshold value is set. When the detection is determined, theprocess proceeds to step 433, otherwise, the process proceeds to step434.

Step 433: the detection notification unit 208 transmits specific personnotification to the terminal device 104. The transmission data includesthe specific person's ID, the person's image, and the like.

Step 434: the determination result storing unit 206 deletes all thestored results. The stored results may not be deleted but overwritten.After the completion of deletion, the process returns to step 400.

As shown in FIG. 5, in stored data 520 in the determination resultstoring unit 206, e.g., a collation result is stored as 1 record, and upto seven records, i.e., the results of 7 times collation are stored inrecords 500 to 506. FIG. 5 shows a status where all the records arevacant.

The records 500 to 506 are respectively configured with a record numbercell 510, a person ID cell 511, a coincidence cell 512, an angle cell513, and a person's region coordinates cell 514.

The record number cell 510 is a region to store a record number. Therecord number is a number used for record management. For example, it isa continuous integer value uniquely assigned by record.

The person ID cell 511 is a region to store a person ID of a mostsimilar person. The person ID is information used for personidentification. For example, it is a character string representing aperson name, an integer value string, a symbol string, or a combinationof these strings, uniquely assigned to the person, or the like.

The coincidence cell 512 is a region to store coincidence.

The angle cell 513 is a region to store an image pickup angle of anappeared person.

The person's region coordinates cell 514 is a region to store thecoordinates of the appeared person's image region.

Here a person's action from appearance to leaving will be moreparticularly described. The appeared person here is an object ofdetection, i.e., a specific person E.

The data shown in FIG. 3 is previously recorded in the specific personrecording unit 200. Further, for the sake of simplification ofexplanation, no other person than the person E appears from theappearance to the leaving of the person E. The image pickup device 101performs image pickup at a rate of 10 frames/sec. The recording device102 records the video image from the image pickup device 101 at a rateof 10 frames/sec. Further, the detection device 103 performs processingon the video image from the image pickup device 101 while performssampling at a rate of 1 frame/sec.

In a timing diagram of FIG. 6, a horizontal axis 600 represents timeseries, and the right side corresponds to the future side. Timings 610to 630 marked with a vertical line on the horizontal axis 600 indicatetiming of the image obtained with the image pickup device 101 andrecorded with the recording device 102. Among them, the timings 610,620, and 630 marked with a symbol Δ under the horizontal axis 600 alsoindicate timing of the image subjected to processing with stationarysampling with the detection device 103. Note that it is assumed that thetiming 620 is time (T). Further, it is assumed that the person E appearsat the timing 614, and leaves at the timing 627. That is, the appearancesection corresponds to the timings 614 to 626.

The content of the processing at the time (T) will be described usingFIGS. 7A and 7B. The reference numerals the same as those in FIGS. 3 and5 respectively denote the same elements.

An image 700 is an image at the time (T), i.e., the image (T) at thetiming 620.

The detection device 103 obtains the image (T) from the image pickupdevice 101 (steps 400 to 402). Then, the detection device performsperson detection and the like on all the regions of the image (steps 403to 404). A detection region 701 indicates the person's region obtainedat step 404. Further, region coordinates 702 indicate coordinate valuesof the detection region 701 in the image. In the present example, anupper left coordinate and a lower right coordinate of the detectionregion 701 are used. The origin is positioned upper left to thedetection region 701 in the figure.

Next, the detection device 103 calculates an image pickup angle withrespect to the detection region 701 (step 405). An angle 703 indicatesthe value calculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the detection region 701 (step 406). A person's feature value704 indicates the value calculated here. As a person's feature value, amultidimensional value is generally used. However, in the presentexample, for the same of simplification of explanation, aone-dimensional small number value is used. The value will be usedherein below.

Next, the detection device 103 performs all collation with respect tothe record data 320 in the specific person recording unit 200, using theperson's feature value 704 (step 407). Collations 710 to 714 indicatecollation with the respective records in the record data 320.Coincidences 720 to 724 indicate coincidence calculated in therespective collations. In the coincidence, as the value is smaller, thesimilarity is higher. As a result of the all collation, the smallestcoincidence is the coincidence 724. Accordingly, it is determined thatthe most similar person is the person E recorded in the record 304. Asthe coincidence, a vector scalar value in person's feature value spaceas a multidimensional value is frequently used. In the present example,for the sake of simplification of explanation, the absolute value of thedifference of the person's feature value is used. The value will be usedherein below.

Next, the detection device 103 compares the coincidence 724 of theperson E as the most similar person obtained above with thepreviously-set first threshold value. In the present example, the firstthreshold value is 1.00. The coincidence 724 is 0.90, which is equal toor lower than the threshold value. Accordingly, as primarydetermination, it is determined that this appeared person is one of thespecific persons, i.e., the person E (step 408).

Next, the detection device 103 stores a detected person ID “E” in a cellcorresponding to the person ID cell 511, the value of the coincidence724 in a cell corresponding to the coincidence cell 512, the value ofthe angle 730 in a cell corresponding to the angle cell 513, the valuesof the region coordinates 702 in a cell corresponding to the person'sregion coordinates cell 514, respectively, with respect to the record500 of the stored data 520 in the determination result storing unit 206(step 409). The stored data 520 in FIG. 7B shows the status of thestored data at this time.

The series of processing in the image 700 is as described above.

The content of processing at the time (T−1) will be described usingFIGS. 8A and 8B.

An image 800 is the image at time (T−1), i.e., the image at the timing619 (T−1).

The detection device 103 obtains the image (T−1) from the recordingdevice 102 (steps 412 to 413). Then the detection device performs persondetection and the like with respect the image (T−1). At this time, theperson detection is performed only on a neighboring region 801 of thedetection region 701. The neighboring region here is a region whichincludes the detection region 701 and which expands vertically andhorizontally from the detection region 701. As the amount of expansion,an appropriate value is previously given in correspondence with apparentmoving speed of the person in the image. In the present example, aregion the same as the detection region 702 is given as the neighboringregion 801. A detection region 802 indicates the person's regionobtained at step 415. Further, region coordinates 803 indicatecoordinate values of the detection region 802 in the image.

Next, the detection device 103 calculates a person's angle with respectto the detection region 802 (step 416). An angle 804 indicates the valuecalculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the detection region 802 (step 417). A person's feature value805 indicates the value calculated here.

Next, the detection device 103 performs collation with respect to therecord data 320 in the specific person recording unit 200 using theperson's feature value 805 (step 418). At this time, the collation isperformed only on the record of the person E as indicated in collation810. Coincidence 820 is coincidence calculated in the collation 810.

Next, the detection device 103 stores a detected person ID “E” in a cellcorresponding to the person ID cell 511, the value of the coincidence820 in a cell corresponding to the coincidence cell 512, the value ofthe angle 804 in a cell corresponding to the angle cell 513, the valuesof the region coordinates 803 in a cell corresponding to the person'sregion coordinates cell 514, respectively, with respect to the record501 of the stored data 520 in the determination result storing unit 206(step 419). The stored data 520 in FIG. 8B shows the status of thestored data at this time.

The series of processing regarding the image 800 is as described above.

The detection device 103 performs the above processing retroactively bythe image at the timing 617 (T−3) or until no person is detected at step414.

The content of processing at steps 422 to 432 will be described usingFIGS. 9A and 9B.

An image 900 is the image at time (T+3) i.e. the image at the timing 623(T+3).

The detection device 103 obtains the image (T+3) from the recordingdevice 102 (steps 423 to 424). Then the detection device performs persondetection and the like with respect to the image. At this time, thedetection is performed only on a neighboring region 901. A detectionregion 902 indicates the person's region obtained at step 426. Further,region coordinates 903 indicate coordinate values of the detectionregion 902 in the image.

Next, the detection device 103 calculates a person's angle with respectto the detection region 902 (step 427). An angle 904 indicates the valuecalculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the detection region 902 (step 428). A person's feature value905 indicates the value calculated here.

Next, the detection device 103 performs collation with respect to therecord data 320 in the specific person recording unit 200 using theperson's feature value 905 (step 429). At this time, the collation isperformed only on the record of the person E as indicated in collation910. Coincidence 920 is the coincidence calculated in the collation 910.

Next, the detection device 103 stores the detected person ID “E” in acell corresponding to the person ID cell 511, the value of thecoincidence 902 in a cell corresponding to the coincidence cell 512, thevalue of the angle 904 in a cell corresponding to the angle cell 513,the values of the region coordinates 903 in a cell corresponding to theperson's region coordinates cell 514, respectively, with respect to therecord 506 of the stored data 520 in the determination result storingunit 206 (step 419). The stored data 520 in FIG. 9B shows the status ofthe stored data at this time.

The series of processing regarding the image 900 is as described above.

Next, the detection device 103 reads the value of the person E's anglepreviously recorded in the specific person recording unit 200. Thedetection device searches for a result of the closest angle from theresults stored in the stored data 520. In the present example, it is therecord 501. Then, the detection device reads the coincidence stored inthe record 501. The detection device compares the read coincidence withthe previously-set second threshold value. In the present example, thesecond threshold value is 0.60. The read coincidence is 0.20, which isequal to or lower than the threshold value. Accordingly, it iscomprehensively determined that the appeared person is one of thespecific persons, and is the person E (step 432).

The series of processing to the detection is as described above.

As it has been shown, in the present example, collation is performed inan image closest to the angle previously recorded in the specific personrecording unit 200. In general, an image having a closer angle has ahigher accuracy. Accordingly, in correspondence with the presentexample, it is possible to obtain higher detection accuracy in aspecific person detection system. Further, in the present example, evenwhen person's images from various angles are not previously registered,it is possible to obtain higher detection accuracy. Further, in thepresent example, even when collation on steady basis is not performed ata high frame rate, it is possible to obtain higher detection accuracy.

In the description, for the sake of simplification of explanation, thenumber of image pickup devices, that of the detection devices, that ofthe recording devices and that of the terminal devices are respectivelyone. However, respectively plural these devices may be connected withthe network.

Further, similarly, respectively one image pickup device and detectiondevice or recording device and detection device oppositely operate,however, it may be configured such that one detection device operatewith respect to plural image pickup devices and recording devices. Onthe contrary, it may be configured such that plural detection devicesoperate with respect to one image pickup device and one recordingdevice.

Further, similarly, respectively one detection device and terminaldevice oppositely operate, however, it may be configured such that oneterminal device operates with respect to plural detection devices. Onthe contrary, it may be configured such that plural terminal devicesoperate with respect to one detection device.

Further, similarly, the image pickup device, the recording device andthe detection device are independent devices, however, they may beimplemented as the same device.

Further, similarly, the detection device and the terminal device areindependent devices, however, they may be implemented as the samedevice.

Further, similarly, the specific person detection system is shown as theobject of the application of the invention, however, the invention maybe implemented with a detection system for not only a specific personbut a specific vehicle, another specific object, or the like, as theobject of application of the invention.

Example 2

Next, the device configuration of the specific person detection systemaccording to an example 2 will be described using FIG. 10. The referencenumerals the same as those in FIG. 1 denote the same devices.

In a specific person detection system 20, image pickup devices 1001 to1003, the recording device 102, the detection device 103, and theterminal device 104 are connected to the network 100, in a mutuallycommunicable status.

The image pickup devices 1001 to 1003 are devices the same as the imagepickup device 101. In the present example, three image pickup devicesare connected to the system. The image pickup devices 1001 to 1003 areinstalled so as to perform image pickup on the same location fromdifferent angles. The detection device 103 stores correspondenceinformation among the image pickup devices 1001 to 1003 performing imagepickup on the same location in a memory device. With this configuration,the detection device 103 grasps the image pickup devices performingimage pickup on the same location.

In the present example, the recording device 102 always records videoimages from the image pickup devices 1001 to 1003.

The flow of processing in the detection device 103 will be describedusing FIGS. 11A to 11C.

Step 1100: the control instruction unit 209 calculates image time (T) ofan image to be obtained next. When specific person detection isperformed by several frames, time several frames ahead of current timeis calculated as the image time (T). Then, the image reception unit 201requests the image pickup device 1001 to output an image (T).

Step 1101: the image reception unit 201 performs image receptionwaiting. When image incoming from the image pickup device 1001 isdetected, the process proceeds to step 1102.

Step 1102: the image reception unit 201 receives an image from the imagepickup device 1001. The image at the reception time (T) is the image(T).

Step 1103: the person detection unit 202 performs person detection withrespect to the received image data. The person detection is performed asin the case of step 403. As a result of above detection, when a personis detected, the process proceeds to step 1104. When a person is notdetected, the process returns to step 1100.

Step 1104: the person detection unit 202 calculates person's regioncoordinates of the person detected at step 1103.

Step 1105: the person's angle calculation unit 203 calculates an anglewith respect to the person's region image calculated at step 1104. Thecalculation of the person's angle is the same as in the case of step405.

Step 1106: the person's feature value extraction unit 204 calculates aperson's feature value with respect to the person's region imagecalculated at step 1104. The person's feature value is the same as inthe case of step 406.

Step 1107: the specific person determination unit 205 performs allcollation on the person's feature value. More particularly, the person'sfeature value calculated at step 1106 is collated (coincidencecalculation) sequentially with respect to all the persons' featurevalues of the respective specific persons previously recorded in thespecific person recording unit 200. A feature value with the smallestcoincidence is found. A person having this feature value is determinedas a most similar person. The coincidence is the same as in the case ofstep 407.

Step 1108: the specific person determination unit 205 performs specificperson detection determination. The detection determination is the sameas in the case of step 408. When detection is determined, the processproceeds to step 1109, otherwise, the process returns to step 1100.

Step 1109: the determination result storing unit 206 stores the resultobtained with respect to the image (T) and saves it. Note that the dataas the object of storage includes the coincidence and the person ID ofthe most similar person obtained at step 1108, the angle obtained atstep 1105, and the coordinates of the person's region calculated at step1104. The aspect of the stored data is the same as that in FIG. 5.

Step 1110: the image reception unit 201 requests the recording device102 to output an image (T) from the image pickup device 1002.

Step 1111: the image reception unit 201 receives the image (T) from theimage pickup device 1002, from the recording device 102.

Step 1112: the person detection unit 202 performs person detection withrespect to the received image data as to the same person as the persondetected at step 1103. The method for person detection is the same as inthe case of step 1103. The identification between the person detected atstep 1103 and the person detected at the present step is the same as inthe case of step 414. Note that in the present example, as the imagepickup position of the image pickup device 1001 and that of the imagepickup device 1002 are different, the installation position relationshipbetween them is previously obtained, and the correspondence between theimage coordinates between the respective images is previously obtainedby geometric calculation. As a result of above detection, when the sameperson is detected, the process proceeds to step 1113. When the sameperson is not detected, the process proceeds to step 1108. The persondetection unit 202 calculates person's region coordinates of the persondetected at step 1112.

Step 1114: the person's angle calculation unit 203 calculates an anglewith respect to the person's region image calculated at step 1113. Themethod for angle calculation is the same as in the case of step 405.

Step 1115: the person's feature value extraction unit 204 calculates aperson's feature value with respect to the person's region imagecalculated at step 1113. The method for calculation of person's featurevalue is the same as in the case of step 406.

Step 1116: the specific person determination unit 205 performs collationon the person's feature value. More particularly, the feature value ofthe most similar person obtained at step 1107 is read from the persons'feature values of the respective specific persons previously recorded inthe specific person recording unit 200. The read feature value iscollated with the person's feature value calculated at step 1115, andcoincidence is calculated. The method for coincidence calculation is thesame as in the case of step 407.

Step 1117: the determination result storing unit 206 stores the resultobtained with respect to the image (T) from the image pickup device 1002and saves it. Note that the data as the object of storage includes thecoincidence obtained at step 1116, the angle obtained at step 1114, andthe coordinates of the person's region calculated at step 1113.

Step 1118: the image reception unit 201 requests the recording device102 to output an image (T) from the image pickup device 1003.

Step 1119: the image reception unit 201 receives the image (T) from theimage pickup device 1003, from the recording device 102.

Step 1120: the person detection unit 202 performs detection with respectto the received image data as to the same person as the person detectedat step 1103. The method for person detection is the same as in the caseof step 1103. The identification between the person detected at step1103 and the person detected at the present step is the same as that atstep 414. Note that in the present example, as the image pickup positionof the image pickup device 1001 and that of the image pickup device 1003are different, the installation position relationship between them ispreviously obtained, and the correspondence between the imagecoordinates between the respective images is previously obtained bygeometric calculation. As a result of above detection, when the sameperson is detected, the process proceeds to step 1121. When the sameperson is not detected, the process proceeds to step 1126.

Step 1121: the person detection unit 202 calculates person's regioncoordinates of the person detected at step 1120.

Step 1122: the person's angle calculation unit 203 calculates an anglewith respect to the person's region image calculated at step 1121. Themethod for angle calculation is the same as in the case of step 405.

Step 1123: the person's feature value extraction unit 204 calculates aperson's feature value with respect to the person's region imagecalculated at step 1121. The method for calculation of person's featurevalue is the same as in the case of step 406.

Step 1124: the specific person determination unit 205 performs collationon the person's feature value. More particularly, the feature value ofthe most similar person obtained at step 1107 is read from the persons'feature values of the respective specific persons previously recorded inthe specific person recording unit 200. The read feature value iscollated with the person's feature value calculated at step 1123, andcoincidence is calculated. The method for coincidence calculation is thesame as in the case of step 407.

Step 1125: the determination result storing unit 206 stores the resultobtained with respect to the image (T) from the image pickup device 1003and saves it. Note that the data as the object of storage includes thecoincidence obtained at step 1124, the angle obtained at step 1122, andthe coordinates of the person's region calculated at step 1121.

Step 1126: the specific person comprehensive determination unit 207performs comprehensive collation using the results stored in thedetermination result storing unit 206 at steps 1109, 1117 and 1125. Thecomprehensive determination is performed by reading the angle of themost similar person obtained at step 1107 from the persons' angles ofthe respective specific persons previously recorded in the specificperson recording unit 200, searching for a result of angle closest tothe angle, among the results stored in the determination result storingunit 206, and in the relation between the coincidence between theresults and a certain constant value (second threshold value), when thecoincidence is equal to or lower than the threshold value,comprehensively determining that the person has been detected as thespecific person. As the second threshold value, a value smaller than thefirst threshold value is set. When the detection is determined, theprocess proceeds to step 1127, otherwise, the process proceeds to step1128.

Step 1127: the detection notification unit 208 transmits the specificperson notification to the terminal device 104. The transmission dataincludes the specific person's ID, the person's image, and the like.

Step 1128: the determination result storing unit 206 deletes all thestored results. The stored results may not be deleted but overwritten.After the completion of deletion, the process returns to step 1100.

The content of processing at the time (T) will be described using FIGS.12A and 12B. The reference numerals the same as those in FIGS. 3 and 5respectively denote the same elements.

An image 1200 is an image (T) from the image pickup device 1001 at thetime (T).

The detection device 103 obtains the image (T) from the image pickupdevice 1001 (steps 1100 to 1102). Then, the detection device performsperson detection and the like with respect to the entire region of theimage (steps 1103 to 1104). A detection region 1201 indicates theperson's region obtained at step 1104. Further, region coordinates 1202indicate coordinate values of the detection region 1201 in the image. Inthe present example, an upper left coordinate and a lower rightcoordinate of the detection region 1201 are used.

Next, the detection device 103 calculates a person's angle with respectto the detection region 1201 (step 1105). An angle 1203 indicates thevalue calculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the detection region 1201 (step 1106). A person's featurevalue 1204 indicates the value calculated here.

Next, the detection device 103 performs all collation with respect tothe record data 320 in the specific person recording unit 200 using theperson's feature value 1204 (step 1107). Collations 1210 to 1214indicate collation with the respective records of the record data.Coincidences 1220 to 1224 indicate coincidences calculated in therespective collations. In the coincidence, as the value is smaller, thedegree of similarity is higher. As a result of all collation, as thesmallest coincidence is the coincidence 1224, it is determined that themost similar person is the person E recorded in the record 304.

Next, the detection device 103 compares the coincidence 1224 of theperson E as the most similar person obtained above with thepreviously-set first threshold value. In the present example, the firstthreshold value is 1.00. The coincidence 1224 is 0.90 which is equal toor lower than the threshold value. Accordingly, it is primarilydetermined that the appeared person is one of the specific persons, andis the person E (step 1108).

Next, the detection device 103 stores the detected person's ID “E” in acell corresponding to the person's ID cell 511, the value of thecoincidence 1224 in a cell corresponding to the coincidence cell 512,the angle 1203 in a cell corresponding to the angle cell 513, the valuesof the region coordinates 1202 in a cell corresponding to the person'sregion coordinates cell 514, respectively, with respect to the record500 of the stored data 520 in the determination result storing unit 206(step 1109). The stored data 520 in FIG. 12B shows the status of thestored data at this time.

The series of processing at steps 1100 to 1109 is as described above.

The content of processing at steps 1110 to 1117 will be described usingFIGS. 13A and 13B.

An image 1300 is an image (T) at the time (T) from the image pickupdevice 1002.

The detection device 103 obtains the image (T) from the image pickupdevice 1002, from the recording device 102 (steps 1110 to 1111). Thedetection device performs person detection and the like with respect tothe entire region of the image (steps 1112 to 1113). A detection region1301 indicates the person's region obtained at step 1113. Further,region coordinates 1302 indicate coordinate values of the detectionregion 1301 in the image.

Next, the detection device 103 calculates a person's angle with respectto the detection region 1301 (step 1114). An angle 1303 indicates thevalue calculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the detection region 1301 (step 1115). A person's featurevalue 1304 indicates the value calculated here.

Next, the detection device 103 performs collation with respect to therecord data 320 in the specific person recording unit 200 using theperson's feature value 1304 (step 1116). At this time, the collation isperformed only on the person E's record as shown in collation 1310.Coincidence 1320 is the coincidence calculated in the collation 1310.

Next, the detection device 103 stores a detected person ID “E” in a cellcorresponding to the person ID cell 511, the value of the coincidence1320 in a cell corresponding to the coincidence cell 512, the angle 1303in a cell corresponding to the angle cell 513, the values of the regioncoordinates 1302 in a cell corresponding to the person's regioncoordinates cell 514, respectively, with respect to the record 501 ofthe stored data 520 in the determination result storing unit 206 (step1117). Stored data 1330 in FIG. 13B shows the status of the stored dataat this time.

The series of processing at steps 1110 to 1117 is as described above.

The content of processing at steps 1118 to 1126 will be described usingFIGS. 14A and 14B.

An image 1400 is an image (T) at the time (T) from the image pickupdevice 1003.

The detection device 103 obtains the image (T) from the image pickupdevice 1003, from the recording device 102 (steps 1118 to 1119). Thedetection device performs person detection and the like with respect tothe entire region of the image (steps 1120 to 1121). A detection region1401 indicates the person's region obtained at step 1121. Further,region coordinates 1402 indicate coordinate values of the detectionregion 1401 in the image.

Next, the detection device 103 calculates a person's angle with respectto the detection region 1401 (step 1122). An angle 1403 indicates thevalue calculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the detection region 1401 (step 1123). A person's featurevalue 1404 indicates the value calculated here.

Next, the detection device 103 performs collation with respect to therecord data 320 in the specific person recording unit 200 using theperson's feature value 1404 (step 1124). At this time, the collation isperformed only on the person E's record as shown in collation 1410.Coincidence 1420 is the coincidence calculated in the collation 1410.

Next, the detection device 103 stores a detected person ID “E” in a cellcorresponding to the person ID cell 511, the value of the coincidence1420 in a cell corresponding to the coincidence cell 512, the angle 1403in a cell corresponding to the angle cell 513, the values of the regioncoordinates 1402 in a cell corresponding to the person's regioncoordinates cell 514, respectively, with respect to the record 502 ofthe stored data 520 in the determination result storing unit 206 (step1125). The stored data 520 in FIG. 14B shows the status of the storeddata at this time.

Next, the detection device 103 reads the person E's angle previouslyrecorded in the specific person recording unit 200. The detection devicesearches for a result of the closest angle from the results stored instored data 1430. In the present example, it is the record 501. Then thecoincidence stored in the record 501 is read. The read coincidence iscompared with the previously-set second threshold value. In the presentexample, the second threshold value is 0.60. The read coincidence is0.20 which is equal to or lower than the threshold value. Accordingly,it is comprehensively determined that the appeared person is one of thespecific persons, and is the person E (step 1126).

The series of processing to the detection is as described above.

As it has been shown, in the present example, collation is performed inan image closest to the angle previously recorded in the specific personrecording unit 200. In general, an image having a closer angle has ahigher accuracy. Accordingly, in correspondence with the presentexample, it is possible to obtain higher detection accuracy in aspecific person detection system. Further, in the present example, evenwhen person's images from various angles are not previously registered,it is possible to obtain higher detection accuracy. It is possible toobtain higher effect than that obtained in the example 1 by installingthe image pickup devices such that the difference of image pickup anglebetween the plural image pickup devices is larger than a directionfluctuation width of a walking person. Further, in the present example,even when collation on steady basis is not performed at a high framerate, it is possible to obtain higher detection accuracy.

In the description, for the sake of simplification of explanation, thenumber of image pickup devices on the same location is three. However,they may be implemented with plural devices other than three.

Further, similarly, the image pickup device, the recording device andthe detection device are independent devices, however, they may beimplemented as the same device.

Further, similarly, the detection device and the terminal device areindependent devices, however, they may be implemented as the samedevice.

Further, similarly, the specific person detection system is shown as theobject of the application of the invention, however, the invention maybe implemented with a detection system for not only a specific personbut a specific vehicle, a specific object, or the like, as the object ofapplication of the invention.

Example 3

Next, the device configuration of the specific person detection systemaccording to an example 3 will be described using FIG. 15. The referencenumerals the same as those in FIG. 1 denote the same devices.

In a specific person detection system 30, image pickup devices 1501 to1504, the recording device 102, the detection device 103, the terminaldevice 104, and a retrieval device 1505 are connected to the network 100in a mutually communicable status.

The image pickup devices 1501 to 1504 are the same as the image pickupdevice 101. In the present example, the four image pickup devices areconnected to the system.

The retrieval device 1505 is a device such as a computer having anarithmetic circuit such as a CPU, a temporary memory such as a RAM, arecording medium such as an HDD, a data transmission bus, an externalinput/output interface, a power source circuit, and the like.

The retrieval device 1505 stores image data from the recording device102, inputted from the network in the external input/output interface,into the temporary memory. The retrieval device performs variousarithmetic operations related to similar image retrieval with respect tothe stored image data using the arithmetic circuit. The recording mediumholds a set of software application programs for similar imageretrieval, an OS (Operation System), a database for storing featurevalues for similar image retrieval and the like. The result of similarimage retrieval is outputted from the external input/output interface tothe network 100.

In the present example, the recording device 102 always records videoimages from the image pickup devices 1501 to 1504. Further, theretrieval device 1505 always collects the video images from the imagepickup devices 1501 to 1504 recorded with the recording device 102,generates a database, in a similar-image retrieval executable status.The details of the retrieval device 1505 will be described later.

FIG. 16 shows an example of the positional relationship of installationof the above four image pickup devices.

As shown in FIG. 16, a passage 1600 indicates a passage on which aperson walks. A route 1601 indicates a route of the person E's walk.Locations 1611 to 1614 respectively indicate the installation positionsof the image pickup devices 1501 to 1504. Image pickup areas 1621 to1624 respectively indicate the range of image pickup with the imagepickup devices 1501 to 1504. The person E walks through the image pickupareas 1621, 1622 and 1623 in this order on the route 1601.

The distance between the image pickup areas is previously measured uponinstallation of the image pickup devices and stored with the controlinstruction unit 209 of the detection device 103.

The aspect of distance data between the image pickup areas in thecontrol instruction unit 209 will be described using FIG. 17.

Distance data 1700 holds distances among the image pickup areas in around-robin manner. In the present example, the distances among fourimage pickup areas are stored. A column 1701 holds the distancesregarding the image pickup area 1621 to other image pickup areas, and acolumn 1702 holds the distances regarding the image pickup area 1622 toother image pickup areas. The columns 1703 and 1704 similarly hold thedistances. A cell 1711 holds the distance between the image pickup areas1621 and the 1622. In the present example, as a distance value, seconds,as required time for walking between the areas are used. Hereinbelow,the same distance value will be used. A cell 1712 holds the distancebetween the image pickup areas 1621 and 1623, and a cell 1713 holds thedistance between the image pickup areas 1621 and 1624, respectively. Inthe present example, required time is used as a stored distance value,however, use of a way (length) or the like may be employed.

The flow of processing in the detection device 103 will be describedusing FIGS. 18A and 18B.

At step 1800, the control instruction unit 209 of the detection device103 calculates image time (T) to be obtained next from the image pickupdevice 1501. When specific person detection is performed by severalframes, time several frames ahead of current time is calculated as imagetime (T). Then, the image reception unit 201 requests the image pickupdevice 1501 to output an image (T).

Step 1801: the image reception unit 201 performs image receptionwaiting. When image incoming from the image pickup device 1501 isdetected, the process proceeds to step 1802.

Step 1802: the image reception unit 201 receives an image from the imagepickup device 1501. The image at the reception time (T) is the image(T).

Step 1803: the person detection unit 202 performs person detection withrespect to the received image data. The person detection is the same asin the case of step 403. As a result of the above, when a person isdetected, the process proceeds to step 1804. When a person is notdetected, the process returns to step 1800.

Step 1804: the person detection unit 202 calculates a person's regioncoordinates of the person detected at step 1803.

Step 1805: the person's angle calculation unit 203 calculates an imagepickup angle with respect to the person's region image calculated atstep 1804. The calculation of person's image pickup angle is the same asin the case of step 405.

Step 1806: the person's feature value extraction unit 204 calculates aperson's feature value with respect to the person's region imagecalculated at step 1804. The person's feature value is the same as inthe case of step 406.

Step 1807: the specific person determination unit 205 performs allcollation on the person's feature value. More particularly, the person'sfeature value calculated at step 1806 is collated (coincidencecalculation) sequentially with respect to all the persons' featurevalues of the respective specific persons previously recorded in thespecific person recording unit 200. A feature value with the smallestcoincidence is found. A person having this feature value is determinedas a most similar person. Note that the coincidence is the same as inthe case of step 407.

Step 1808: the specific person determination unit 205 performs specificperson detection determination. The detection determination is the sameas in the case of step 408. When detection is determined, the processproceeds to step 1809, otherwise, the process returns to step 1800.

Step 1809: the determination result storing unit 206 stores the resultobtained with respect to the image (T) and saves it. Note that the dataas the object of storage includes the coincidence and the person ID ofthe most similar person obtained at step 1808, the angle obtained atstep 1805, and the coordinates of the person's region calculated at step1804. The aspect of the stored data is the same as in the case of FIG.5.

Step 1810: the control instruction unit 209 refers to the distance data1700, and calculates a proximity image pickup area of the image pickupdevice 1501 that obtained the image (T). The proximity image pickup areais an image pickup area with a distance to the image pickup device 1501that obtained the image (T) in short-distance relationship equal to orlower than a certain constant value. There may be plural proximity imagepickup areas. Further, although not shown in FIG. 18, when a proximityimage pickup area does not exist at step 1810, the process proceeds tostep 1822.

Step 1811: predicted arrival time (TF) to the farthest image pickup areafrom the proximity image pickup area, obtained at step 1810 of theperson detected at step 1803, is calculated. The predicted farthestarrival time is obtained by calculation from the distance value storedin the distance data 1700.

Step 1812: the control instruction unit 209 waits by the time (TF).After the arrival of the time (TF), the process proceeds to step 1813.

Step 1813: the control instruction unit 209 requests the retrievaldevice 1505 to perform similar image retrieval with respect to theperson's region of the image (T) calculated at step 1804. At this time,as a condition regarding image pickup time, the time from the time (T)and before the time (TF), and as a condition regarding image pickuparea, the proximity image pickup area obtained at step 1810, and thesimilarity equal to or higher than a predetermined constant value, aregiven as narrowing conditions for the similar image retrieval.

Step 1814: the control instruction unit 209 receives a retrieval resultfrom the retrieval device 1505. The retrieval result includes an image(R) and coordinates of a person's region in the image. Note that R ise.g. an integer value when the number of images included in theretrieval result is five and R=1 to 5 holds. Further, 5, as a maximumvalue of R, is stored as a retrieval result number S.

Step 1815: the control instruction unit 209 resets a counter (R) to 0.

Step 1816: the control instruction unit 209 compares the counter (R)with the retrieval result number (S). When R is less than S, the processproceeds to step 1817. When R is equal to or greater than S, the processproceeds to step 1822.

Step 1817: the person's angle calculation unit 203 calculates an imagepickup angle with respect to the person's region obtained from the image(R) and its person's region obtained at step 1814. The method for anglecalculation is the same as in the case of step 405.

Step 1818: the person's feature value extraction unit 204 calculates aperson's feature value with respect to the person's region obtained fromthe image (R) and its person's region coordinates obtained at step 1814.The method for calculation of person's feature value is the same as inthe case of step 406.

Step 1819: the specific person determination unit 205 performs collationon the person's feature value. More particularly, the feature value ofthe most similar person obtained at step 1807 is read from the persons'feature values of the respective specific persons previously recorded inthe specific person recording unit 200. The specific persondetermination unit collates the read feature value with the person'sfeature value calculated at step 1818, and calculates coincidence. Themethod for coincidence calculation is the same as in the case of step407.

Step 1820: the determination result storing unit 206 stores the resultobtained with respect to the image (R) and saves it. Note that the dataas the object of storage includes the coincidence obtained at step 1819,and the image pickup angle obtained at step 1817.

Step 1821: the control instruction unit 209 increments the counter (R)Lby 1, and returns the control to step 1816.

Step 1822: the specific person comprehensive determination unit 207performs comprehensive determination using the results stored in thedetermination result storing unit 206 at steps 1809 and 1820. Thecomprehensive determination is performed by reading the image pickupangle of the most similar person obtained at step 1807 from the imagepickup angles of the respective specific persons previously recorded inthe specific person recording unit 200, searching for a result of imagepickup angle closest to the image pickup angle, among the results storedin the determination result storing unit 206, and in the relationbetween the coincidence between the results and a certain constant value(second threshold value), when the coincidence is equal to or lower thanthe threshold value, comprehensively determining that the person hasbeen detected as the specific person. As the second threshold value, avalue smaller than the first threshold value is set. When the detectionis determined, the process proceeds to step 1823, otherwise, the processproceeds to step 1824.

Step 1823: the detection notification unit 208 transmits the specificperson notification to the terminal device 104. The transmission dataincludes the specific person's ID, its person image, and the like.

Step 1824: the determination result storing unit 206 deletes all thestored results. The stored results may not be deleted but overwritten.After the completion of deletion, the process returns to step 1800.

The action when the object of detection, i.e., the person E as thespecific person, walks on the route 1601 in the passage 1600 will bemore particularly described. The person E appears in the image pickuparea 1621 at the time (T). Further, the data shown in FIG. 3 ispreviously recorded in the specific person recording unit 200. Further,for the sake of simplification of explanation, no other person than theperson E appears from the appearance to the leaving of the person E.

The content of processing at the time (T) will be described using FIGS.19A and 19B. The reference numerals the same as those in FIGS. 3 and 5respectively denote the same elements.

An image 1900 is an image (T) obtained by image pickup with respect tothe image pickup area 1621 with the image pickup device 1501 at the time(T).

The detection device 103 obtains the image (T) from the image pickupdevice 1501 (steps 1800 to 1802). Then, the detection device performsperson detection and the like with respect to the entire region of theimage (steps 1803 to 1804). A detection region 1901 indicates theperson's region obtained at step 1804. Further, region coordinates 1902indicate coordinate values of the detection region 1901 in the image. Inthe present example, an upper left coordinate and a lower rightcoordinate of the detection region 1901 are used.

Next, the detection device 103 calculates an image pickup angle withrespect to the detection region 1901 (step 1805). An image pickup angle1903 indicates the value calculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the detection region 1901 (step 1806). A person's featurevalue 1904 indicates the value calculated here.

Next, the detection device 103 performs all collation with respect tothe record data 320 in the specific person recording unit 200, using theperson's feature value 1904 (step 1807). Collations 1910 to 1914indicate collation with the respective records in the record data.Coincidences 1920 to 1924 indicate coincidence calculated in therespective collations. In the coincidence, as the value is smaller, thesimilarity is higher. As a result of the all collation, the smallestcoincidence is the coincidence 1924. Accordingly, it is determined thatthe most similar person is the person E recorded in the record 304.

Next, the detection device 103 compares the coincidence 1924 of theperson E as the most similar person obtained above with thepreviously-set first threshold value. In the present example, the firstthreshold value is 1.00. The coincidence 1924 is 0.90 which is equal toor lower than the threshold value. Accordingly, it is primarilydetermined that the appeared person is one of the specific persons, andis the person E (step 1808).

Next, the detection device 103 stores the detected person's ID “E” in acell corresponding to the person's ID cell 511, the value of thecoincidence 1924 in a cell corresponding to the coincidence cell 512,the angle 1903 in a cell corresponding to the angle cell 513,respectively, with respect to the record 500 of the stored data 520 inthe determination result storing unit 206 (step 1809). The stored data520 in FIG. 19B shows the status of the stored data at this time.

The series of processing at steps 1800 to 1809 is as described above.

Next, the detection device 103 calculates a proximity image pickup areaof the image pickup area 1621. In this example, in the distance data1700, the column 1701 holding the distance to the image pickup area 1621is compared to the cells 1711 to 1713 in this order, to obtain storedvalues. The obtained stored values are compared with a predeterminedthreshold value, to obtain a proximity image pickup area existing in adistance equal to or lower than the threshold value, i.e., a shortdistance. There may be plural proximity image pickup areas. In thepresent example, assuming that the threshold value is 20, and as aresult, the image pickup areas 1622, 1623 and 1624 are selected asproximity image pickup areas (step 1810). Then, (T+18), as a total ofthe stored value of the image pickup area 1624, as an image pickup areahaving the longest distance among the selected three image pickup areas,and the time (T), becomes predicted arrival time (TF) (step 1811). Inthe present example, the stored value of the image pickup area 1624 issimply added, however, a margin may be further added.

Next, the detection device 103 waits by the time (T+18) (step 1812).

The content of processing at the time (T+18) will be described usingFIGS. 20A to 20C. The reference numerals the same as those in FIGS. 3and 5 respectively denote the same elements.

The detection device 103 requests the retrieval device 1505 to performsimilar image retrieval. The retrieval key at that time is the detectionregion 1901. As narrowing conditions regarding the image pickup time,from the time (T), and before the time (T+18) are given. Further, asnarrowing conditions regarding the image pickup area, the image pickupareas 1622, 1623 and 1624 selected at step 1810 are given (step 1813).

A retrieval result 2000 indicates a retrieval result received at step1814. In the present example, the retrieval result includes two imagesand coordinates of person's region in the respective images. An image2010 is the first image in the retrieval result, and an image 2020 isthe second image in the retrieval result. Further, a person's region2011 is the person's region in the image 2010, and a person's region2021 is the person's region in the image 2020.

Next, the detection device 103 calculates an image pickup angle withrespect to the person's region 2011 (step 1817). An image pickup angle2012 indicates the value calculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the person's region 2011 (step 1818). A person's featurevalue 2013 indicates the value calculated here.

Next, the detection device 103 performs collation with respect to therecord data 320 in the specific person recording unit 200 using theperson's feature value 2013 (step 1819). At this time, the collation isperformed only on the person E's record as indicated in collation 2030.Coincidence 2040 is coincidence calculated in the collation 2030.

Next, the detection device 103 stores the detected person ID “E” in acell corresponding to the person ID cell 511, the value of thecoincidence 2040 in a cell corresponding to the coincidence cell 512,and the angle 2012 in a cell corresponding to the angle cell 513,respectively, with respect to the record 501 of the stored data 520 inthe determination result storing unit 206 (step 1820). The stored data520 in FIG. 20B shows the status of the stored data at this time.

Next, the detection device 103 calculates an image pickup angle withrespect to the person's region 2021 (step 1817). An image pickup angle2022 indicates the value calculated here.

Next, the detection device 103 calculates a person's feature value withrespect to the person's region 2021 (step 1818). A person's featurevalue 2023 indicates the value calculated here.

Next, the detection device 103 performs collation with respect to therecord data in the specific person recording unit 200 using the person'sfeature value 2023 (step 1819). At this time, the collation is performedonly on the person E's record as indicated in collation 2060.Coincidence 2070 is coincidence calculated in the collation 2060.

Next, the detection device 103 stores the detected person's ID “E” in acell corresponding to the person's ID cell 511, the value of thecoincidence 2070 in a cell corresponding to the coincidence cell 512,and the angle 2022 in a cell corresponding to the angle cell 513,respectively, with respect to the record 502 of the stored data 520 inthe determination result storing unit 206 (step 1820). The stored data520 in FIG. 20C shows the status of the stored data at this time.

Next, the detection device 103 reads the person E's angle previouslyrecorded in the specific person recording unit 200, and searches for aresult of the closest angle from the results stored in the stored data5200 in FIG. 20C. In the present example, it is the record 501. Then,the coincidence stored in the record 501 is read. The read coincidenceis compared with the previously-set second threshold value. In thepresent example, the second threshold value is 0.60. The readcoincidence is 0.20 which is equal to or lower than the threshold value.Accordingly, it is comprehensively determined that the appeared personis one of the specific persons and is the person E (step 1822).

The series of processing to the detection is as described above.

As shown in FIG. 21, the retrieval device 1505 has parts, an imagerecording unit 2101, an image feature value recording unit 2102, animage feature value extraction unit 2103, an image similaritydetermination unit 2104, and a face search unit 2105.

The image recording unit 2101 is a part to record image data inputtedfrom the image pickup devices 1501 to 1504 or the recording device 102into an unshown recording medium such as an HDD. Upon recording of theimage data, the image recording unit 2101 simultaneously recordsinformation to read the image data later, e.g. a frame numbersequentially assigned on the frame basis from the start of recording asa unique number in the recording device. The frame number is a numberwhich is sequentially assigned upon continuous storage of image bypredetermined period as in the case of a moving image such as a videoimage. Further, the image recording unit 2101 also records informationto discriminate when the image has been picked up, e.g., image time,simultaneously. At this time, the image time is e.g. device timeoutputted from a clock included in the recording device 102, or devicetime outputted from a clock respectively included in the image pickupdevices 1501 to 1504. Further, the image recording unit 2101 alsorecords information to discriminate the image pickup device thatobtained the image, e.g., an IP address of the image pickup device,simultaneously.

The image feature value recording unit 2102 is a part to record an imagefeature value into the recording medium. The image feature valuerecording unit 2102 first obtains face image data by outputting theimage data recorded with the image recording unit 2101 to the facesearch unit 2105. Then, the image feature value recording unit 2102obtains an image feature value by outputting the face image data to theimage feature value extraction unit 2103. Upon recording of the imagefeature amount, the image feature value recording unit 2102simultaneously records a frame number corresponding to the image datainputted in the face search unit 2105. Hereinbelow, list data formedwith the frame numbers and image feature values generated with thisrecording will be referred to as “image feature value list data”.

The image feature value extraction unit 2103 is a part to calculate afeature value of the face image data inputted from the face search unit2105 using an image recognition technique. As the image feature value,e.g., color distribution, edge pattern composition distribution of theimage, or a combination of them is used.

The image similarity determination unit 2104 is a part to perform imageretrieval and output a retrieval result. The image similaritydetermination unit 2104 calculates similarity from the image featurevalue of a retrieval image described below, and the image feature valueof the face image in the image data recorded in the recording medium ofthe image recording unit 2101, and generates a retrieval result from thedegree of the calculated similarity. The retrieval image is an imagedesignated with the control instruction unit 209 in the detection device103 as an model image inputted and referred to for similaritydetermination. The retrieval image is inputted as data included in aretrieval request signal. More particularly, it is designated with aframe number or the like of the image. Note that the image feature valueof the retrieval image is obtained with the image similaritydetermination unit 2104 by outputting to the face search unit 2105.Further, the image feature value of the face image data recorded in therecording medium is obtained from the above-described image featurevalue list data recorded in the image feature value recording unit 2102.Further, the method for image similarity calculation is configured byreferring to papers such as “Visualization Models for Large Image Sets”(HIROIKE, Atsushi et.al, Journal of Japan Photo Society 2003, vol. 66no. 1, P 93-P 101).

When a retrieval request signal from the control instruction unit 209 ofthe detection device 103 is received, a controller of the retrievaldevice 1505 inputs a retrieval image designated with the controlinstruction unit 209 of the detection device 103 into the face searchunit 2105, to search for a face. When a face is detected, the imagefeature value extraction unit 2103 extracts the image feature value of aface region included in the retrieval image. Then, the image similaritydetermination unit 2104 compares the above image feature value with thefeature values in the image feature value list data previously extractedwith registration processing. Image similarity is determined based onthe image feature value. The record image in the recording device issearched for a person's image who a user desires to find by execution ofretrieval processing, by using frame information or the like included inthe image feature value list data. With this conventional registrationprocessing, it is possible to realize similar face image retrieval witha monitor image as the object.

As it has been shown, in the present example, collation is performed inan image closest to the angle previously recorded in the specific personrecording unit 200. In general, an image having a closer angle has ahigher accuracy. Accordingly, in correspondence with the presentexample, it is possible to obtain higher detection accuracy in aspecific person detection system. Further, in the present example, evenwhen person's images from various angles are not previously registered,it is possible to obtain higher detection accuracy. Even when imagepickup has not been performed on the same location with plural imagepickup devices as in the case of the example 2, by utilizing acombination of image pickup devices in multiple locations, it ispossible to obtain a high effect approximately equivalent to thatobtained in the example 2.

In the description, for the sake of simplification of explanation, thenumber of image pickup devices is four. However, they may be implementedwith plural devices other than four.

Further, similarly, the image pickup device, the recording device andthe detection device are independent devices, however, they may beimplemented as the same device.

Further, similarly, the detection device and the retrieval device areindependent devices, however, they may be implemented as the samedevice.

Further, similarly, the detection device and the terminal device areindependent devices, however, they may be implemented as the samedevice.

Further, similarly, the specific person detection system is shown as theobject of the application of the invention, however, the invention maybe implemented with a detection system for not only a specific personbut a specific vehicle, a specific object, as the object of applicationof the invention.

As described above, the invention made by the present inventor has beenparticularly explained based on the embodiments and examples, however,the present invention is not limited to the above-described embodimentsand examples. It goes without saying that various changes can be made.

REFERENCE SIGNS LIST

10, 20, 30: specific person detection system, 100: network, 101: imagepickup device, 102: detection device, 103: recording device, 104:terminal device, 200: specific person recording unit, 201: imagereception unit, 202: person detection unit, 203: person's anglecalculation unit, 204: person's feature value calculation unit, 205:specific person determination unit, 206: determination result storingunit, 207: specific person comprehensive determination unit, 208:detection notification unit, 209: control instruction unit, 300 to 304:record, 310: record number cell, 310, 311: person's ID cell, 312:feature value cell, 313: angle cell, 314: person image cell, 320: recorddata, 400 to 434: step, 500 to 506: record, 510: record number cell,511: person's ID cell, 512: coincidence cell, 513: angle cell, 514:person's region coordinates cell, 520: stored data, 600: horizontalaxis, 610 to 630: timing, 700: image, 701: detection region, 702: regioncoordinates, 703: angle, 704: person's feature value, 710 to 714:collation, 720 to 724: coincidence, 800: image, 801: neighboring region,802: detection region, 803: region coordinates, 804: angle, 805:person's feature value, 810: collation, 820: coincidence, 900: image,901: neighboring region, 902: detection region, 903: region coordinates,904: angle, 905: person's feature value, 910: collation, 920:coincidence, 1001 to 1003: image pickup device, 1110 to 1127: step,1200: image, 1201: detection region, 1202: region coordinates, 1203:angle, 1204: person's feature value, 1210 to 1214: collation, 1220 to1224: coincidence, 1300: image, 1301: detection region, 1302: regioncoordinates, 1303: angle, 1304: person's feature value, 1310: collation,1320: coincidence, 1400: image, 1401: detection region, 1402: regioncoordinates, 1403: angle, 1404: person's feature value, 1410: collation,1420: coincidence, 1501 to 1504: image pickup device, 1505: retrievaldevice, 1600: passage, 1601: route, 1611 to 1614: location, 1621 to1624: image pickup area, 1700: distance data, 1701 to 1704: column, 1711to 1713: cell, 1800 to 1824: step, 1900: image, 1901: detection region,1902: region coordinates, 1903: angle, 1904: person's feature value,1910 to 1914: collation, 1920 to 1924: coincidence, 2000: retrievalresult, 2010: image, 2011: person's region, 2012: angle, 2013: person'sfeature value, 2020: image, 2021: person's region, 2022: angle, 2023:person's feature value, 2030: collation, 2040: coincidence, 2060:collation, and 2070: coincidence.

1. A specific person detection system comprising: a detection deviceincluding a specific person recording unit that holds a specific person,an image pickup device, and a terminal device, wherein the detectiondevice obtains a person most similar to a feature value extracted fromimage data obtained by the image pickup device from the specific personrecording unit, calculates similarity between a feature value extractedfrom other image data and the person, and outputs a person having angleinformation most similar to the person, from persons having highsimilarity, as a collation result to the terminal device.
 2. Thespecific person detection system according to claim 1, furthercomprising a recording device recording the other image data.
 3. Thespecific person detection system according to claim 2, wherein thedetection device stores a person's image pickup angle of the person in alist of specific persons, detects an arbitrary person with his/her imagepickup angle, from an image obtained with the image pickup device,obtains coincidence from feature all collation between the list ofspecific persons and the detected person, and performs specific personprimary detection, obtains other image than the primary-detected imagefrom the recording device, detects a person the same as aprimary-detected person with his/her image pickup angle, from theobtained other image, and performs collation, and performs specificperson comprehensive determination using the coincidence by a series ofcollation including the primary-detected result and the image pickupangle.
 4. The specific person detection system according to claim 3,wherein the other image is an image obtained at time near the primarydetection with the same image pickup device.
 5. The specific persondetection system according to claim 2, wherein the detection devicestores a list of specific persons, detects an arbitrary person from anobtained image, obtains coincidence from feature collation between thelist of specific persons and the detected person, and detects specificperson.
 6. The specific person detection system according to claim 1,further comprising a other image pickup device obtaining the otherimage.
 7. The specific person detection system according to claim 6,wherein the detection device stores an image pickup angle of the personin a list of specific persons, detects an arbitrary person, with his/herimage pickup angle, from the image obtained with the one image pickupdevice, obtains coincidence from feature all collation between the listof specific persons and the detected person, and performs primarydetection of a specific person, obtains other image than theprimary-detected image is obtained from the other image pickup device,detects a person the same as a primary-detected person, with his/herimage pickup angle, from the obtained other image, and performscollation, and performs specific person comprehensive determinationusing the coincidence by a series of collation including theprimary-detected result and the image pickup angle.
 8. The specificperson detection system according to claim 7, wherein the other image isan image obtained with the other image pickup device at the same time asthe time when the image obtained with the image pickup device isobtained.
 9. The specific person detection system according to claim 6,wherein the detection device stores a list of specific persons, detectsan arbitrary person from an obtained image, obtains coincidence fromfeature collation between the list of specific persons and the detectedperson, and detects specific person.
 10. The specific person detectionsystem according to claim 6, wherein the detection device furtherobtains an image pickup device near the image pickup device, and whereinthe other image is an image in which the same person appears, obtainedby similar image retrieval with the primary-detected image as a key,under conditions of near time and near location.
 11. The specific persondetection system according to claim 10, further comprising a retrievaldevice, wherein the retrieval device retrieves the other image.
 12. Thespecific person detection system according to claim 10, wherein thedetection device stores a list of specific persons, detects an arbitraryperson from an obtained image, obtains coincidence from featurecollation between the list of specific persons and the detected person,and detects specific person.
 13. A specific person detection methodcomprising: recording a specific person in a specific person recordingunit; obtaining a person most similar to a feature value extracted fromimage data from the specific person recording unit; calculatingsimilarity between a feature value extracted from other image data andthe person; and outputting a person having angle information mostsimilar to the person, from persons having high similarity, as acollation result.
 14. A detection device comprising: a specific personrecording unit that holds a specific person; a specific persondetermination unit that performs primary determination as to whether ornot a specific person, using a feature value; and a specific personcomprehensive determination unit that performs secondary determinationfrom plural determination results. wherein the specific persondetermination unit obtains a person most similar to a feature valueextracted from image data from the specific person recording unit, andcalculates similarity between a feature value extracted from other imagedata and a feature value of the person, and the specific personcomprehensive determination unit outputs a person having angleinformation most similar to the person, from persons having highsimilarity, as a collation result.